ROX-31495: wiremock for central #32
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #32 +/- ##
=======================================
Coverage 77.36% 77.36%
=======================================
Files 26 26
Lines 1162 1162
=======================================
Hits 899 899
Misses 223 223
Partials 40 40 ☔ View full report in Codecov by Sentry. |
Implements a WireMock-based mock service to enable development and testing without requiring an actual StackRox Central instance. Features: - Standalone Java-based WireMock service (no Docker required) - gRPC support via WireMock gRPC extension - Token-based authentication validation - Parameter-based response mappings for different CVE queries - Easy-to-edit JSON fixture files - Automated setup scripts for downloading JARs and generating proto descriptors - Make targets for service lifecycle management (start/stop/restart/status/logs) - Comprehensive smoke test suite - CI integration via GitHub Actions Scripts: - scripts/download-wiremock.sh: Download WireMock JARs from Maven Central - scripts/setup-proto-files.sh: Copy proto files from stackrox repo - scripts/generate-proto-descriptors.sh: Generate proto descriptors for gRPC - scripts/start-mock-central.sh: Start WireMock service - scripts/stop-mock-central.sh: Stop WireMock service - scripts/smoke-test-wiremock.sh: Run comprehensive smoke tests Make targets: - make mock-download: Download WireMock JARs - make mock-start/stop/restart: Control service lifecycle - make mock-status: Check service status - make mock-logs: View service logs - make mock-test: Run smoke tests Test scenarios included: - CVE-2021-44228 (Log4j): Returns 3 affected deployments - CVE-2024-1234: Returns 1 custom deployment - Authentication: Validates Bearer tokens (test-token-*) - Empty queries: Returns empty results CI Integration: - Automated smoke tests run on PRs touching WireMock files - Verifies all required files are committed - Tests WireMock setup, authentication, CVE queries, and MCP integration - Uploads logs on failure for debugging Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> # Conflicts: # .gitignore
- Removed file existence checks (unnecessary - tests fail anyway if files missing) - Reduced from 28 tests to 7 focused integration tests - Test flow: setup → start WireMock → test endpoints → verify MCP integration - Added cleanup trap for better resource management - 30 lines shorter and more maintainable Tests now verify: 1. WireMock starts and runs 2. Admin API works 3. Authentication validation works 4. CVE queries return correct data 5. MCP server can connect with WireMock config All 7 tests passing.
- Remove unnecessary comments and verbose logging from all scripts - Simplify README sections and fix outdated smoke test documentation - Update CI workflow to run on all PRs (removed path-based triggers) - Reduce code verbosity while maintaining functionality All smoke tests passing (7/7). Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Replace manual proto file copying from ../stackrox repository with automated
approach using Go mod cache, following the stackrox repository pattern.
Changes:
- Update setup-proto-files.sh to use `go list -f '{{.Dir}}' -m` for module discovery
- Get proto files from github.com/stackrox/rox module
- Get scanner protos from github.com/stackrox/scanner module
- Add Makefile targets: proto-setup, proto-generate, proto-clean, proto-check
- Simplify GitHub Actions workflow (removed external repo checkout)
- Update documentation to reflect new approach
- Add proto-version.sh script for version tracking
Benefits:
- No external repository dependencies
- Works automatically in CI/CD environments
- Version-locked to go.mod for reproducibility
- Handles read-only mod cache files with chmod
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
janisz
force-pushed
the
ROX-31495-wiremock-mock-service
branch
from
54e2b8f to
4b7b1e4
Compare
Implemented comprehensive E2E testing framework with complete eval coverage: - Test runner supports --mock and --real flags - Mock mode: WireMock with TLS (self-signed cert) - Real mode: staging.demo.stackrox.com - Automatic WireMock lifecycle management - Self-signed certificate generation (wiremock/generate-cert.sh) - HTTPS on port 8081 with proper TLS - Uses InsecureSkipTLSVerify (no client code changes needed) - Idempotent cert generation with keytool dependency check - Added 3 new test tasks: log4shell, multiple CVEs, RHSA - Total 11 E2E tests with proper assertions - 32/32 assertions passing - 5 new fixtures for E2E test CVEs - 3 deployment fixtures (CVE-2021-31805, CVE-2016-1000031, CVE-2024-52577) - 2 cluster fixtures (CVE-2016-1000031, CVE-2021-31805) - Updated mappings with CVE-specific routing Modified: - .gitignore - Added wiremock/certs/ exclusion - e2e-tests/README.md - Mock/real mode documentation - e2e-tests/mcpchecker/eval.yaml - Added 3 new tests - e2e-tests/scripts/run-tests.sh - Mock/real mode switching - scripts/start-mock-central.sh - TLS configuration - wiremock/README.md - Updated fixture documentation - wiremock/mappings/clusters.json - CVE-specific mappings - wiremock/mappings/deployments.json - CVE-specific mappings Created: - e2e-tests/mcpchecker/tasks/cve-log4shell.yaml - e2e-tests/mcpchecker/tasks/cve-multiple.yaml - e2e-tests/mcpchecker/tasks/rhsa-not-supported.yaml - e2e-tests/scripts/smoke-test-mock.sh - wiremock/fixtures/deployments/cve_2021_31805.json - wiremock/fixtures/deployments/cve_2016_1000031.json - wiremock/fixtures/deployments/cve_2024_52577.json - wiremock/fixtures/clusters/cve_2016_1000031.json - wiremock/fixtures/clusters/cve_2021_31805.json - wiremock/generate-cert.sh - IMPLEMENTATION_SUMMARY.md - All shellcheck issues resolved - Proper error handling and dependency checks - Idempotent operations throughout - Clean TLS approach (no client code modifications) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> Signed-off-by: Tomasz Janiszewski <tomek@redhat.com>
janisz
force-pushed
the
ROX-31495-wiremock-mock-service
branch
from
4b7b1e4 to
cf2af0c
Compare
This commit implements proper configuration for E2E tests to run against the WireMock mock service instead of requiring a real StackRox instance. Key Changes: - Created mcp-config-mock.yaml with explicit environment variables (fixes mcpchecker's inability to inherit env vars properly) - Created eval-mock.yaml that references the mock config - Updated run-tests.sh to select correct eval file based on mode - Added HTTP port 8080 to WireMock startup for debugging - Updated cluster mappings to include CVE-2099-00001 and CVE-2024-52577 - Fixed cluster fixture data to match test expectations WireMock gRPC Configuration: - Proto descriptors must use .dsc extension (handled by setup scripts) - JSON fixtures are automatically converted to protobuf by gRPC extension - Removed explicit Content-Type headers to let extension handle encoding Test Results: - 10 out of 11 tests passing (91% pass rate) - All tool invocation assertions passing (29/32) - Tests no longer hang - proper connection to mock service - Remaining failures are LLM behavioral (tool call counts), not mock issues Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit fixes the remaining 2 E2E test failures by correcting WireMock request matching for gRPC requests. Root Causes Fixed: 1. **JSONPath pattern mismatch**: The mappings used $.query[?(@.query...)] which looked for a nested array structure, but gRPC protobuf-to-JSON conversion creates a simple object with a "query" field (lowercase). 2. **Cluster name inconsistency**: Test asked for "staging-central-cluster" but it wasn't in the general cluster list, only in CVE-specific files. Changes: - Updated all CVE mappings from $.query[?(@.query =~ ...)] to $[?(@.query =~ ...)] to match actual protobuf JSON structure - Added "staging-central-cluster" to all_clusters.json for consistency with cve_2016_1000031.json Key Insight: Protobuf field names use lowercase (as defined in .proto files), while Go field names use PascalCase. When gRPC requests are converted to JSON by WireMock's gRPC extension, they use the protobuf field name "query", not the Go field name "Query". Test Results: - All 11/11 tests passing (100% pass rate) ✅ - All 32/32 assertions passing ✅ - cve-nonexistent: Now correctly returns empty cluster list - cve-cluster-does-exist: Now finds cluster and checks CVE Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Mock central with WireMock
Validation
CI